Why not just use your mail server's spam filter?

It is usually running already, and that is exactly the problem: you cannot see what it removes. Most setups put suspect mail in a separate folder your system never opens. On top of that, a filter like this does not know your customers. The practical answer is not to choose but to combine: read the server's spam folder as well, and put your own safe list of customers and partners above it.

Does a self-learning filter not go wrong as soon as somebody marks something incorrectly?

You cover that risk with three limits. The learning layer learns only from a marking made by a human, never from its own verdict. It only starts counting once there is enough material, in our case at least fifty spam and fifty good messages. And its maximum weight in the final score is capped, so a run of skewed markings cannot hijack the whole thing.

How long do you keep what has been marked as spam?

Three months in our case, with four safeguards around it. Anything a human has touched stays. The daily digest warns seven days before anything is cleaned up. Every clean-up round is logged. And the term only starts running after the filter has spent a period merely observing, because otherwise you clear out what you cannot yet judge.

A spam filter for your shared mailbox

Spam is a nuisance. A filter that quietly discards a booking request costs more. How to turn that starting point into four layers and three outcomes.

A shared mailbox fills up. The address on your website receives all sorts: enquiries, invoices, newsletters nobody signed up for, and the familiar junk.

The reflex is a spam filter. Understandable, but the real risk is on the other side. A filter that removes a booking enquiry once a month costs you more than all the spam put together. And you do not notice, because you simply never hear from that customer again.

One sentence decides the whole design

The client put it this way at the start: rather let a bit of spam through by accident than lose a real email.

That is not a weak compromise. It is a technical choice with a name. You optimise for a low chance of a false positive, meaning good mail that ends up as spam, and you accept more false negatives in return, meaning spam that gets through. Every threshold, every fallback behaviour and every retention period follows from that one sentence.

It pays to write that down before you start. Otherwise you end up with a filter that looks sharp and that nobody dares to rely on.

What the audit turned up first

Before a single line of filter code was written, three things surfaced that were already wrong regardless of spam.

The fetch read one folder, and that was the inbox. Whatever the mail server itself flagged as spam went to a separate folder the system never opened. So a filter was already running that nobody could see. That is exactly the scenario you want to avoid, and it was already the case.

The header fields were discarded on import. Only sender, recipient, subject, date and message number survived. The very fields a verdict should rest on, the outcome of SPF, DKIM and DMARC, the sending address and the mail server's own verdict, disappeared. Without those fields every decision is guesswork on text alone.

And "spam" was being stored as a label. A conversation has one label, so anyone marking spam wiped the subject label. A spam state belongs in a field of its own.

Four layers, in this order

The first layer is a safe list, and it always wins. Never spam: everyone in the address book, every customer, every user of the system whatever their role, every address we have ever emailed ourselves, and every conversation that already contains a reply from us. That is the cheapest and strongest protection against losing real mail, and the data was already in house. The list is looked up again at every assessment, because a customer who books today belongs on it tomorrow.

The second layer is the hard facts from the header. No opinion, only what is verifiable: the outcome of SPF, DKIM and DMARC, the verdict the mail server already passed, the sending address against the known block lists and a list of trusted senders, and the markers of bulk mail.

The third layer is the learning layer: word statistics over your own mail. This is the part that makes "self-learning" real, and the only part that can learn your exceptions. Mail from a particular foreign government body is spam here and is not spam for the rest of the world. No off-the-shelf filter will ever say that.

The fourth layer is the AI, and only as a referee in the doubt zone. Its weight is capped and it can never put a message in the spam folder on its own. The mail goes to the model as content and never as an instruction. If the connection is down, the layer simply drops away and the other three decide.

Three outcomes instead of two

This is the core. Most filters know two outcomes: through or gone. This filter knows three.

Below the low threshold nothing happens. Above the high threshold the mail moves to the spam folder. And in between sits a doubt band in which the mail stays in the queue, with a grey flag reading "possibly spam" and the reason beside it.

That middle outcome is the whole point. A filter that has to choose between discarding and doing nothing discards too often. A filter that is allowed to hesitate can be cautious without becoming useless.

Two weeks of watching first

For the first two weeks the filter was not allowed to move anything. Only to assess, record and show what it would have done.

That is how grown-up filters are rolled out, and there is a second gain in it: you find your own thresholds. What counts as a clear spam score at one organisation is the normal score at another for a supplier who sends their mail through a bulk service.

Keep the header, including what you do not use yet

One technical choice that is worth gold later: the whole header is kept, not just the fields in use today.

Email is temporary. If you want to add an extra signal six months from now, you can only verify it against mail you still hold. With the whole header in house you test a new signal against your own history, without fetching anything again.

The trap we found along the way

A sender is free to put a line of authentication results in their own mail. That makes such a line worthless on its own: anyone can write down that their own mail passes the check.

So only the top line counts, because that is where your own mail server puts its line, and the name of that server is stored separately. If it is not your own mail server, the outcome says nothing and nothing is done with it. The same goes for the route a message travelled: only the top lines are examined, and internal addresses do not count.

What it gives you

A cleaner inbox, but that is not the main prize.

The main prize is that you see the reason behind every verdict. A message with a flag and the text "sender fails the check on the sending domain" is something a colleague can make a decision about. A message that disappears unnoticed is not.

And a human marking always wins, permanently. If someone marks it Not spam, it stays that way, and the filter learns from it.

Rather let a bit of spam through than lose a customer email