How Supervised Machine Learning Can Stop Spear-Phishing

A spear-phishing email that works passes SPF, DKIM and DMARC. It has to — the attacker registered the domain, published the records, and signed the mail properly, because doing so is free and takes ten minutes. Authentication tells you a message genuinely came from the domain it claims. It says nothing whatsoever about whether that domain should be trusted, and a decade of “check the sender is authenticated” user training has taught people to treat a green tick as a verdict it was never designed to give.

That is the gap. Secure email gateways were built for spam: high-volume, low-effort campaigns that leave reusable fingerprints across millions of messages. A campaign sent to four people, written specifically for those four people, from clean infrastructure, has no fingerprint to reuse. There is nothing statistical to catch and nothing on a blocklist to match.

Supervised machine learning is the usual answer, and it is a genuinely good one — with caveats this article spends as much time on as the benefits, because a classifier deployed inline on your mail flow is a system that can silently destroy legitimate business email. That failure mode deserves more attention than it normally gets.

Throughout, this article uses a running illustrative scenario — a mid-sized bank with a compromised transfer-gateway credential — to make the pipeline concrete. It is a composite for teaching purposes, not a report on a specific institution.

Why Spear-Phishing Is So Dangerous

The preparation is the weapon. An attacker will spend weeks profiling a target from LinkedIn, press releases, conference talks, GitHub commits and quarterly filings before writing a word. By the time the email lands it references a real project by its internal name, mentions a colleague who genuinely sits two desks away, and matches the register the recipient expects from that sender. The recipient is not being careless when they act on it. They are being competent, on the basis of accurate context.

Three specific properties make these attacks hard to catch:

Hijacked trust is the worst of them. An attacker who has compromised one mailbox and replies within an existing thread inherits every trust signal that thread has accumulated — correct subject line, correct quoted history, correct participants. Nothing about the message is anomalous, because structurally it is a genuine reply. Detection has to come from how the message is written, not from where it came from.

Micro-targeting keeps volume deliberately below the threshold at which volume-based detection works. Ten recipients across a week produces no spike, no burst, no bulk-mail signature. Every control keyed to campaign scale is blind by construction.

Manufactured urgency does the psychological work. A deadline tied to a board meeting or a senior executive compresses the window in which someone might stop and verify. This is the one durable signal across almost all spear-phishing, which is why urgency features carry so much weight in the models discussed below — and also why they generate false positives every quarter-end, when genuine urgent finance email spikes.

These manipulation tactics are covered in depth in our Social Engineering — Complete Roadmap. Our guide to Building a 24/7 Tier-1 SOC in Malaysia also covers how structured shift design reduces analyst fatigue — another factor attackers actively exploit.

Why Traditional Filters Fail

Conventional secure email gateways work on static indicators: known-bad file hashes, blocklisted URLs, sending IP reputation, domain age, bulk-mailing behaviour, and heuristics for trigger phrases or malformed HTML. Against commodity spam this is extremely effective and extremely cheap, which is why it is still the first layer everywhere and should stay that way. Nothing here argues for removing it.

It fails against targeted mail for a structural reason: every input it consumes is under the attacker’s control and costs almost nothing to make clean. A domain aged sixty days, warmed with legitimate traffic, correctly authenticated, hosting a payload built for one recipient, produces no match anywhere. The tool is not broken. It is being asked a question it was never designed to answer.

The deeper limitation is that deterministic rules have no notion of relationship. They cannot tell you that this sender has emailed this recipient forty times and never once used this phrasing, or that a thread’s register changed abruptly at message six. That relational and stylistic context is precisely what a trained classifier can represent — with the significant caveat that it can only represent what your historical data actually contains.

Supervised Machine Learning for Spear-Phishing Detection

The concept is simple: train on thousands of emails already labelled benign or malicious, and the algorithm learns a boundary between them. The hard part is never the algorithm. It is the labels.

Where do labels for spear-phishing come from? Mostly from attacks you already caught — which means the training set is a record of the attacker techniques your existing controls were good enough to detect. The messages that got through are, by definition, absent or mislabelled as benign. Every supervised email classifier inherits this bias, and it is why user-reported phishing is disproportionately valuable training data: it is the only reliable source of examples that defeated the automated layer.

A practical pipeline for phishing email classification typically runs through these stages:

1. Ingestion — Raw EML or MIME objects pulled from the mail flow, before any gateway rewriting mangles URLs and headers you need.
2. Feature Extraction — Header anomalies, lexical patterns in the body, sender–recipient relationship metrics.
3. Vectorisation — Text and structural data converted to numeric form via TF-IDF, one-hot encoding, or dense embeddings.
4. Model Training — Random Forest, linear SVM, or gradient boosting (XGBoost) fitted on the extracted features.
5. Validation — k-fold cross-validation, split by time rather than randomly, plus adversarial inputs.
6. Deployment — Inline at the MTA, or as an enrichment step feeding the SIEM.

Step 5 is where most projects quietly go wrong. Randomly shuffling a corpus before splitting lets messages from the same campaign land in both training and test sets, and the model gets credit for recognising a campaign it has already memorised. Split chronologically — train on everything before a date, test on everything after — and reported performance usually drops sharply. That lower number is the real one.

The genuine advantage of supervised learning here is locality: your organisation’s own traffic teaches the model what normal looks like for your users, your vendors, your writing conventions. The corresponding cost is that the model is not portable, degrades when the business changes — a merger, a new region, a new CRM sending on your behalf — and needs an owner who notices.

Model / Algorithm	Primary Strengths	Notable Trade-offs
Random Forest	Handles mixed feature types without much preprocessing; built-in importance metrics make triage explicable to analysts.	Memory footprint grows with the forest; impurity-based importances are misleading on high-cardinality features — use permutation importance instead.
Linear SVM	Strong margins on sparse text features; weight vectors are directly readable.	Needs careful tuning, and it gives you no calibrated probability, so risk thresholds have to be derived separately.
XGBoost	Usually the best raw accuracy on tabular email features; fast at inference.	Overfits happily on small label sets, and its ensemble structure means SHAP is not optional if analysts must justify a quarantine decision.

import email
import re
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble import RandomForestClassifier

# Sample raw email data and corresponding labels (1 = Phishing, 0 = Benign)
sample_emails = [
    "Subject: Urgent: Verify your SWIFT credentials\n\nDear team, please verify your SWIFT gateway details immediately on our portal.",
    "Subject: Q3 Project Board Review meeting notes\n\nHi John, here are the minutes from the Q3 project board review meeting. Let me know if you have feedback."
]
labels = [1, 0]

def preprocess(raw_email_str):
    msg = email.message_from_string(raw_email_str)
    subject = msg['Subject'] or ''
    body = ''
    
    if msg.is_multipart():
        for part in msg.walk():
            if part.get_content_type() == 'text/plain':
                payload = part.get_payload(decode=True)
                if payload:
                    body += payload.decode(errors='ignore')
    else:
        payload = msg.get_payload(decode=True)
        if payload:
            body = payload.decode(errors='ignore')
            
    cleaned_text = re.sub(r'[^A-Za-z]+', ' ', subject + ' ' + body).lower()
    return cleaned_text

processed_emails = [preprocess(mail) for mail in sample_emails]
vectorizer = TfidfVectorizer(ngram_range=(1, 2), max_features=20000)
X = vectorizer.fit_transform(processed_emails)

classifier = RandomForestClassifier(n_estimators=100, n_jobs=-1, class_weight='balanced', random_state=42)
classifier.fit(X, labels)

print(f"Model successfully trained on {X.shape[0]} samples with {X.shape[1]} features.")

Run as a sidecar to the MTA, inference on a model this size costs single-digit milliseconds — the model is not what makes mail slow. Be sceptical of the F1-scores above 0.98 that get quoted for this architecture: those come from balanced public corpora, where malicious and benign are roughly equal in number. Your inbound mail is not balanced, and on real traffic the same model will look considerably worse. The published figure is a sanity check that the pipeline works, not a forecast of production performance.

Logging every feature vector alongside the verdict is the part worth insisting on. It gives investigators a reconstructable record of why a message was flagged, and it is the only way to diagnose the failure mode that matters — a model that has started scoring differently without anyone changing it, because the traffic underneath it moved.

Feature Engineering and Public Datasets

Feature engineering is where the actual security expertise goes. The model does not know what SPF is; it knows that column 47 correlates with the label. Someone has to decide that column 47 should exist, and that decision is a threat-modelling judgement dressed up as data preparation.

Feature Category	Example Metric / Feature	Cyber Threat Indicator / Rationale
Header Integrity	Mismatch between From and Return-Path, or SPF/DMARC failure.	Catches lazy spoofing. Note that competent spear-phishing passes all of these, so this family has high precision and poor recall.
Authentication Alignment	DKIM verification status and alignment with the visible From domain.	Useful chiefly for detecting misconfigured legitimate senders, which is most of what it will actually flag.
Semantic Analysis	Density of imperative verbs, urgency tokens, monetary cues.	The strongest single family for targeted mail — and the one that misfires on genuine finance and legal correspondence.
HTML Structure	Hidden iframes, remote CSS imports, form input tags in body content.	Credential-harvesting pages and tracking pixels. Newsletters trip this constantly; allowlist your marketing platforms.
Relationship Baseline	Has this sender ever emailed this recipient before, and at what frequency?	The most durable feature in practice, because an attacker cannot fabricate a history they were never part of.
Behavioural Tone Shift	Deviation from the sender’s own historical writing style.	The only family that detects a compromised internal account, where every other signal is legitimate by construction.

Three public corpora are the usual starting points, each with a limitation worth stating plainly. The Enron Email Dataset gives roughly half a million real corporate messages as a benign baseline — from a single company, in 2001, before HTML mail, cloud SaaS notifications or modern business writing conventions. The Nazario Phishing Corpus provides labelled historical phishing, weighted heavily toward commodity campaigns rather than targeted ones. PhishTank supplies a live community-maintained feed of phishing URLs, excellent for URL features and useless for everything else.

Together they will get a pipeline working end to end. None of them will teach a model what spear-phishing against your organisation looks like, and a model trained only on public data and then deployed inline is the most common way this project fails visibly.

[!TIP] Store the raw MIME, the extracted feature vector and the final label together, in append-only storage with restricted write access. When someone asks in six months why a message was quarantined, the model that made the decision will have been retrained several times — the vector is the only record of what the classifier actually saw. Retention here is subject to the same data-protection rules as the mailboxes themselves; agree the period with legal before you start collecting.

From Lab to Production: Operational Checklist

A notebook model and a component sitting in the delivery path of your company’s email are different things carrying very different risk. The notebook can be wrong. The inline classifier being wrong means a customer’s contract never arrived and nobody knows why.

Start in monitor-only mode. Score every message, log every verdict, quarantine nothing, and run that way for at least a full business cycle. What you are looking for is not accuracy — it is the list of legitimate senders the model dislikes, and that list is always longer and stranger than expected.

Ingestion hardening. Capture the stream with Postfix’s always_bcc or a Milter, before URL rewriting alters the content you want to analyse. Hash or drop personal data you do not need for features; a phishing pipeline that quietly becomes a permanent archive of all corporate email is a data-protection problem waiting for an auditor.

Lifecycle governance. Pick a retraining cadence — 30 days through MLflow is a reasonable default — and treat every retrain as a deployment: versioned, evaluated against a held-out chronological set, and reversible. Keep a rollback path to the previous model artefact, because you will need it, and the moment you need it will be during a delivery incident.

Security controls. Read-only root filesystems on inference containers, signed commits for configuration, restricted access to training data. Note that the training pipeline is itself an attack surface: an adversary who can get chosen messages labelled benign — by sending mail that a helpdesk routinely marks as safe — can shift a decision boundary over time. Poisoning is slow, quiet, and does not look like an attack in any log you keep.

Feedback loops. Weekly analyst triage of false positives and negatives, fed back as labels. Without it the model degrades, and the degradation is invisible: precision stays acceptable while recall falls, so your dashboards look fine and more attacks get through. Track recall against user-reported phishing specifically, since that is the closest thing you have to an unbiased measure of what you missed.

For containerisation and security isolation guidance, our guide on setting up a home web server on Raspberry Pi 5 covers relevant isolation patterns in a practical context.

Evaluating Model Performance: Beyond Basic Accuracy

Targeted phishing is a fraction of a percent of inbound mail. At that ratio a classifier that marks everything benign scores well above 99% accuracy, so accuracy is not a weak metric here — it is an actively misleading one, and it is still what gets put on the slide.

Three measures earn their place:

F1-Score balances precision and recall in one number, which makes it a reasonable summary and a poor decision tool. It weights a missed spear-phish and a quarantined invoice equally. Your business does not.

Matthews Correlation Coefficient (MCC) holds up far better under imbalance, returning a high value only when the model does well on both classes at once. Majority-class guessing scores near zero. If you report one number, report this one.

Campaign-Level AUROC groups attempts by payload hash or sender metadata and asks whether coordinated waves get caught, rather than scoring each message in isolation. Catching one message of a ten-message campaign and missing nine is a failure that per-message metrics will happily describe as 10% recall and move on from.

Concrete numbers help, so take the running bank scenario: a Random Forest reaching roughly 0.96 F1 against a rule-based gateway at 0.71 on the same traffic. That gap is real and typical. Read it carefully, though — a 0.96 F1 at that base rate still means a steady trickle of quarantined legitimate mail, and someone in the business will be on the receiving end of it every week. Budget for that person’s time.

SHAP values are what makes the difference between an analyst who can close a ticket and one who is guessing. Being able to say a detection was driven by an unfamiliar sender relationship plus urgency phrasing gives the reviewer something to check. It also gives the recipient of a false positive an explanation, which is the difference between a security control people tolerate and one they route around.

Open-Source Tooling for Your Security Pipeline

Operational Area	Recommended Tool	Core Value / Practical Notes
Feature Extraction	Apache Tika	Parses over a thousand attachment formats. It is also parsing hostile files, so sandbox it — Tika has had its own CVEs, and it runs on exactly the input an attacker chooses.
Text Vectorisation	Scikit-learn (TF-IDF)	The right baseline. Fit the vectoriser on training data only; fitting on the full corpus leaks test information and inflates every number you report.
Model Training	XGBoost / LightGBM	Strong on large sparse feature sets with low inference cost. LightGBM trains faster; XGBoost is better documented for this use case.
ML Pipeline Ops	MLflow	Parameter tracking, model registry, versioning. The registry matters more than the tracking — it is what makes rollback a command rather than an archaeology exercise.
Threat Intelligence	PhishTank API	Live URL feed for link features. Community-submitted, so treat entries as a signal rather than ground truth, and rate-limit accordingly.

For detailed instructions on setting up a self-hosted container environment to run these models, see our Portainer Stack Tutorial.

Deploying Models in Modern SOCs

Sitting the classifier behind the MTA lets you act during delivery rather than after. A score at or above some threshold — 0.80 is a common starting point, and it is a business decision rather than a technical one — triggers a SOAR playbook:

1. Ticket Creation — A high-priority incident in Jira or ServiceNow with the feature vector attached.
2. Retroactive Hunting — Search mailboxes for similar messages delivered over the previous 30 days and purge them.
3. Intel Sharing — Push malicious IPs, domains and hashes to the threat intelligence platform.

Step 2 is the one that repays the whole build, and also the one that deserves a guardrail. Automated retroactive purge is a capability to delete mail from arbitrary mailboxes on the strength of a model score. Get the threshold wrong, or let a poisoned model shift, and you have built a very efficient mechanism for destroying legitimate correspondence across the organisation. Require human approval above a blast radius — say, ten mailboxes — and log every purge with its justification.

Explainable output is what lets analysts verify rather than guess, and it is the difference between a queue that gets worked and a queue that gets acknowledged. It is also increasingly what an auditor expects when an automated system takes an action against a person’s communications.

If you’re building a career in this space, our Ultimate Guide to SOC & SIEM Careers covers the full landscape. We also break down the platform differences in SIEM vs. SOAR: Which One Do You Need?.

MITRE ATT&CK Mapping & Regulatory Compliance

Mapping detections to a shared framework is mostly about making your coverage arguable. This control addresses MITRE ATT&CK T1566.001 (Spearphishing Attachment) and T1566.002 (Spearphishing Link) — and notably does not address T1566.003 (Spearphishing via Service), where the lure arrives over LinkedIn or WhatsApp and never touches your mail flow at all. Claiming full T1566 coverage from an email classifier is the kind of overreach that gets found in an assessment.

On the regulatory side, an adaptive classifier contributes to incident management expectations under ISO/IEC 27001 and to technology risk requirements such as the MAS TRM Guidelines for firms regulated in Singapore. Contributes, not satisfies — no auditor accepts a model as a control on its own. What they want is documentation of how it is governed: who owns it, how it is retrained, how decisions are explained, and what happens when it is wrong. Build that record while you build the pipeline, because reconstructing it a year later is far more work.

Next-Generation Research in Email Security

Four directions are worth tracking, with the usual caveat that research results and production results are different animals:

Graph Neural Networks model the organisation’s communication as a graph, so trust becomes a structural property — who talks to whom, how often, through which intermediaries. An attacker on fresh infrastructure has no position in that graph and cannot manufacture one. The catch is that a compromised internal account has a perfect position in it, which is precisely the case you most want to catch.

Contrastive learning pre-trains on large unlabelled corpora, then fine-tunes on a small set of high-quality labelled attacks. This directly addresses the label scarcity problem described earlier, and is probably the most practically promising item on this list for organisations that will never accumulate thousands of confirmed spear-phishing samples.

Federated learning allows several organisations to improve a shared model without exchanging email content. Attractive on paper, and the honest position is that it is not yet a solved deployment: model updates can leak information about training data, and the governance question of who owns the resulting model is usually harder than the technical one.

Edge inference compiles scoring models into lightweight runtimes inside the MTA, cutting latency and allowing a verdict before delivery. The trade-off is operational — a model embedded in the mail server is harder to update, harder to roll back, and couples your mail infrastructure’s release cycle to your detection work.

What all four have in common is that they need someone who understands both the modelling and the threat. Teams that hire only data scientists build accurate classifiers of the wrong thing.

Retrospective Case Study: Pacific Rim Bank

Returning to the composite bank scenario, here is what a working inline classifier looks like as a timeline. The point of walking through it is the gaps between the rows — the two minutes of scoring, and the seven minutes before a human looked at it:

Time (UTC+8)	Incident Event	Active Detection / Security Layer
09:02 AM	A highly targeted email titled “Q1 Governance Metrics” is delivered.	Mail Transfer Agent (MTA)
09:04 AM	The inline ML classifier scores the message at 0.87 risk and routes it to quarantine.	Supervised ML Model
09:05 AM	An automated SOAR playbook executes, running threat intelligence checks.	SOAR Integration
09:12 AM	A security analyst reviews the quarantined mail; the embedded URL is confirmed as a credential harvester.	Analyst / Malware Sandbox
09:18 AM	The SOC runs a retroactive mailbox query and confirms zero delivery across the organisation.	SIEM / Log Analytics
10:44 AM	A secondary payload delivery attempt is blocked at the proxy; audit logs are secured.	Web Proxy / Digital Forensics

Two features carried most of the score, and neither is exotic. Missing or malformed X-Mailer and related client headers, which automated sending frameworks routinely omit and real clients routinely set. And an attachment-type deviation against the sender’s own history — this correspondent had only ever sent spreadsheets. Neither feature is individually convincing. Together, on a sender with a thin relationship history, they cleared the threshold.

Notice what the classifier did not do. It did not prevent the initial delivery attempt, identify the attacker, or determine scope; a human did all three between 09:12 and 09:18. What the model bought was those nine minutes, and nine minutes is the entire difference between quarantining a credential harvester and running a forensic investigation into who clicked it. That is the honest value proposition — not that machine learning stops spear-phishing, but that it moves the human into the loop early enough to matter.

The same detection then feeds back as a training label, which is the compounding part and the reason the weekly triage discipline is worth protecting when everyone is busy.

Summary: Staying Ahead of the Threat

Supervised learning has stopped being experimental for email defence, but it is a layer rather than a replacement. The gateway still catches the commodity volume far more cheaply than any model will. The classifier catches what the gateway structurally cannot, and it does so at the cost of a feedback loop somebody has to run every week, a false-positive queue somebody has to work, and a governance record somebody has to maintain.

If you take one thing from this: the project’s success is determined by label quality and by chronological evaluation, not by model selection. Teams spend months choosing between XGBoost and a neural network and then split their data randomly, and the resulting system performs nothing like the number in the deck.

Spear-phishing is not becoming more technically sophisticated so much as better researched, and better research is not something a mail filter can fix. What a good classifier does is shorten the window between arrival and a human paying attention. That is a smaller claim than the marketing makes, and it is worth building.

---

Further Reading

• AI in Cybersecurity: Revolutionizing Threat Detection and Response
• Why SMS 2FA Is No Longer Safe
• Self-Hosting Lemmy: A Secure, Decentralized Social Media Platform