I know exactly the spam you are speaking about.

Have you considered the Spamhaus RBL? It is very effective, particularly if you use the zen.spamhaus.org (do not enable received header analysis though).

Thanks for the response, Elric. I knew there was something I should have mentioned. I do use the block lists, mainly spamcop and spamhaus, and this does have a good hit rate but I'm not sure that I'm using that one so will look into it. Ideally I'd like to just block post/web master emails but I know that isn't allowed.

I also mentioned in another post that the auto-whitelist icon doesn't appear to be working when using the SpamAssasin plug-in on its own. Any chance you could investigate that?

http://forum.mailtraq.com/viewtopic.php?p=2869#2869

I'm currently using sbl-xbl.spamhaus.org but I see that zen.spamhaus.org is now recommended. I can't seem to disable the received headers for just this DBL though so if I turn it off completely how will that affect my other DBL configurations? Presumably they might miss matches that might have otherwise be picked up?

I'll turn off received header analysis for now but would appreciate any guidance on this.

JayC wrote:

> I'll turn off received header analysis for now but would appreciate any guidance on this

Leave it turned off permanently.

There was a time, a few million Internet years ago, when header information could be trusted - sort of. These days, unless you already know by other means that the message source is trusted, it's crazy to even think of basing any decision-making processes on ip addresses extracted from Received headers which (may) have been forged by net-abusers.

Typically, there are a maximum of two Received headers which you can trust. You can always trust the first one because Mailtraq takes great pains to ensure that it's accurate. If you receive all your mail from a service provider, for example, you should also be able to rely on the Received header which they've generated. Thereafter, it's impossible to tell whether or not the remaining Received headers have been forged.

I think I must have configured that a few million Internet years ago! I really must take the time to review all of Mailtraq's settings as I have to be honest and say I'm guilty of the "set it and forget it" syndrome. Hopefully using the combined ZEN check should help. I also use SpamCop, SORBS, and the combined NJABL.

Thanks for the response, especially as I know how much you hate forums.

Is there a document that gives the steps to setting all of this up, as well as managing things?

I too am finding that the bayes learning is not as effective as it used to be. I am getting so many good messages being captured that I'm ready to chuck the whole antispam thing and simply let the users deal with it all. I don't know what changed, but I've never had this much problem before, and I have not changed anything. I'm spending way too much time going over the captured mail in the "spam" folder looking for real messages then copying them to the intended recipient. It is a clumsy process at best. :(

There probably is a guide somewhere (Edit: I've been told it is here). As is the current fashion, I suggest reading Wikipedia's DNSBL entry. (DBL is a contraction of DNSBL which is itself a contraction of DNS Black List, which is itself a contraction of...)

In my experience, Spamhaus have done a fantastic job and their lists are proving to be very effective. To support any DNS Black list you really just need the list host name. Some organisations, like Spamhaus, have a number of lists to suit your tastes. zen.spamhaus.org is Spamhaus's SBL, XBL and PBL combination and you install it into Mailtraq like this :-

1. Go to Options | Services...[/*:m]
2. Open the properties for your SMTP service[/*:m]
3. Go to the Black Lists tab and you will see your currently configured list of DBL hosts[/*:m]
4. Click Add[/*:m]
5. In the DNS Lookup Expression you want: %4.%3.%2.%1.zen.spamhaus.org (The prefix just means that Mailtraq replaces the placeholders with the IP address in reverse. Basically all lists work the same way)[/*:m]
   
6. In Response Handling and Interpreting you can leave the default. If you want to know exactly what kind of rejections are being used, copy my screenshot[/*:m]

You can also use bl.spamcop.net which has also come a long way and I now trust it. After using these two, you should see a huge spam reduction.

Also, make sure that you are rejecting non-local senders claiming to be local (if your network configuration supports it). This enabled by clicking on Explicit Black Lists and going to the Senders. You must, in conjunction, ensure that only IP addresses listed in your Local network (Options | Server... | LAN) are allowed to send mail from your domains. Also enable Using local HELO arguments on the HELO Hosts tab.

The above is important because spam software is designed to get several million e-mails out as fast as possible and using the receiver's host name and domain name will most likely save them the time it takes while the receiver does a reverse lookup. You'd think it was to bypass checks (which it is, to some degree), but really it is all about speed.

Hopefully the above will help you.

All that said, I still don't think you are managing spam the right way. You are training, and accepting mail, on behalf of your recipients when they should really be training, and rejecting mail, by themselves. I understand why you are doing this, but going against standard methods will always be difficult. As we revise the WebMail interfaces, however, we will keep your requirements in mind.

[/]

Attachment	Size
LOG_10.TXT	3.99 MB
LOG_09.TXT	5.6 MB
imap movistar.txt	2.26 KB
imap movistar.txt	2.26 KB
imap.txt	48.03 KB
mark_olszewski.txt	1.39 MB
configuration_Server01.zip	67.21 KB
configuration_Server02.zip	58.59 KB

As always, your response and counsel is wonderful and appreciated. Two challenges we have prevent us from using some of this. The first is that all our users employ Outlook, thus cannot train for spam without using webmail. The second being that we employ a proxy, and the first line of every email shows the email came from our proxy's IP. I know there is probably a tweak for this, but am not sure how to proceed. The beginning of every SMTP connection from the proxy (in the logs) shows the connection coming from a whitelisted address, thus the connection is automatically allowed. I'm wondering how this may be affecting my ability to stop the flood of spam.

7/18/2007 6:14 PM SMTP: (Accept) Receiving connection from 192.168.1.99
7/18/2007 6:14 PM SMTP (0) IP Whitelist: Matched 192.168.1.99

A good point. One of my colleagues is working on a method for training with IMAP by moving messages to and from the Junk Mail folder. I have an idea for an Outlook extension for the future. But of course none of this helps you.

But in the mean time, you aren't alone. This is quite a common configuration and normally what our users do is the following :-

• Have all mail forwarded to another mail slot which shares the same database, but which the administrator can access via WebMail. Instead of completely rejecting mail, mark all mail with something in the subject that includes the score (I think that's the default) and then configure Outlook to move mail with that subject line to a spam folder. [/*:m]
• What the administrator does is occasionally log on via WebMail and trains any errors since the lost logon. The users' databases benefit from this and eventually Mailtraq becomes more effective and errors become less common. [/*:m]
• When you reach a more comfortable level (when very few false positives are above the 0.05 score) you switch on the "Reject on Receipt" option and set it to 0.05. Users are encouraged to keep an eye on their spam folder, but set Outlook to archive it sooner for performance.[/*:m]

The problem with spam is that it is distracting, more than anything else. Generally users are comfortable receiving spam if it says "This is spam" in the subject and is put somewhere separate.

Yes this is a major issue. We recommend against letting any product other than a transparent hardware proxy receive your mail before Mailtraq because it will interfere with many of Mailtraq's defenses. It is quite capable of being your first defense for SMTP.

You can switch on "Check the xth IP address" and set that value to "1" and that will help. What proxy software is it? Our forthcoming gateway product doesn't suffer this problem because it passes on the original IP address to Mailtraq.

[/]

Interesting, in fact, I was going to request something like that, but thought it too far out. I used to work on a MS Exchange network that used the Espion Interceptor antispam system. We could drag a spam message to a shared folder that the interceptor would learn as spam.

We're currently using QBik's Wingate. Were using Winproxy, but it recently went end of life, so had to jump ship after many years.

Is it running on the same machine as Mailtraq or on a different machine?

We have it running on a different PC.

I haven't used wingate for several years but a v6 document (p38), suggests that transparent pass through (NAT & port forwarding) would allow all of Mailtraq's SMTP defences to be available. Under ENS Properties, Port Range Configuration I'd expect Don't translate source IP to let Mailtraq see connections from the outside world, not the proxy. (With this setup, Mailtraq's default gateway would have to be the Wingate machine). It's worth considering -- if you're on v6...

Cheers, Martin

Martin,

Thanks for the heads up on that. I actually overlooked that translate IP checkbox. I made a snip of my MTQ log before and after the change was made. Instead of processing every inbound connection, it is now rejecting all the connections outright. I'm not sure what's up with that (found that the firewall on MTQ was set to only allow connections on option #3, local IP's. I had to set it to option #1 to make things work. Since I don't allow relaying unless they are authenticated, I guess that is ok).

7/19/2007 8:09 AM SMTP: (Accept) Receiving connection from 192.168.1.99
7/19/2007 8:09 AM SMTP (0) IP Whitelist: Matched 192.168.1.99
7/19/2007 8:09 AM SMTP (915) HELO kkoxk ---> 250 faith-outreach.org
7/19/2007 8:09 AM SMTP (915) MAIL From: ---> 250 receiving from ixzlh@cannonsburg.com
7/19/2007 8:09 AM Bouncing mail from ixzlh@cannonsburg.com to lafrenie@faith-outreach.org
7/19/2007 8:09 AM SMTP (915) RCPT TO: ---> 550 mailbox not found
7/19/2007 8:09 AM SMTP (915) QUIT ---> 221 have a nice day (SMTP Closing)
7/19/2007 8:09 AM SMTP (915) SMTP Client Disconnected (192.168.1.99): No DATA
7/19/2007 8:11 AM SMTP: (Firewall) Denying connection to 213.213.236.210
7/19/2007 8:12 AM SMTP: (Firewall) Denying connection to 151.23.132.102
7/19/2007 8:12 AM SMTP: (Firewall) Denying connection to 24.193.189.100
7/19/2007 8:12 AM SMTP: (Firewall) Denying connection to 216.22.49.195
7/19/2007 8:12 AM SMTP: (Firewall) Denying connection to 151.44.129.100
7/19/2007 8:12 AM POP3: (Accept) Receiving connection from 192.168.1.53
7/19/2007 8:12 AM awalker: Logged In via POP3
7/19/2007 8:13 AM SMTP: (Firewall) Denying connection to 82.194.107.108
7/19/2007 8:13 AM SMTP: (Firewall) Denying connection to 65.77.131.177
7/19/2007 8:13 AM POP3: (Accept) Receiving connection from 192.168.1.52
7/19/2007 8:13 AM bserna: Logged In via POP3
7/19/2007 8:13 AM SMTP: (Firewall) Denying connection to 58.143.108.66
7/19/2007 8:13 AM SMTP: (Firewall) Denying connection to 86.126.229.50
7/19/2007 8:13 AM SMTP: (Firewall) Denying connection to 201.13.102.40

Check the firewall settings on the SMTP service (and any other service that's receiving NAT/port-forwarded traffic) - normally, you'd need to allow any client to connect.

It would also be a good time to review your SMTP service, Relaying settings ('Relay for non-local senders' is the only safe option).

Cheers, Martin

Wanted to let everyone know that setting up the blacklist like Elric said, and changing Wingate to not translate the incoming IP from the proxy server made TONS of difference. Where we were getting 500 spams through to our SPAM folder in 24 hours, we are getting 25. I'm still training the database so spams don't get through to the user mailslots, but the difference is night and day. Had some problems with rejections early on, but fixed that. Had to not include the PBL entries in the blacklist because even my email ISP was rejected. Just using SBL and XBL is enough to reduce things nicely and not reject too much. The number of spams being rejected because of being on the spamhaus list is unbelievable. We're also rejecting emails because of invalid HELO addresses as well.

Appreciate your help, Martin and Elric. This forum is fantastic.